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ABSTRACT 

A major activity of any ESEA Title III project is 
evaluation. This paper suggests evaluation methods especially 
appropriate tc such projects by applying a systens approach to the 
evaluation design. Evaluation as a system is divided into three 
subsystems: (1) baseline evaluation, which describes conditions as 

they exist before project treatment; (2) process evaluation, which 
provides infcrmation to aid decisionmaking in day-to-day operational 
activities; and (3) product evaluation, which measures the extent to 
which a set cf project objectives has been met. These three 
evaluatior. ethods provide information to facilitate decisionmaking 
during t!'.e course of the project, instead of postponing evaluation 
until after ccmpletior^ of the project- This type of continuous 
evaluation system should be an important part of any innovation 
program. (Figure 4 on page 30 may reproduce poorly because of 
marginal legibility.) (RA) 
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INTRODUCTION 

Ttic purpose of this study is to develop an evaluation system 
with specific application to projects approved and funded under 
Title III of the Elementary and Secondary Education Act of 1965* 
This evaluation system has within itself sufficient flexibility 
to provide for differences among the various projects subsumed 
under Title III ESEA. At the same time it is rooted in the 
scientific method of investigation and is developed by using the 
systems approach. 

One specific objective of this study is to develop a valid 
rationale for systematic evaluation and subsequently plate it in 
perspective with other project-related activities. Evaluation for 
ESEA Title 111 projects is depicted as a set of subsystems forming 
pal t of a larger system --- the project. Another objective of 
this study Is to Illustrate how the system approach can be applied 
effectively and logically to the evaluation of ESEA Title III 
projects and all project-related activities. 

Throe distinct subsystems for evaluation are developed and 
woven together into the larger system. Tnese three phases of 
evaluation arc baseline evaluation , process evaluation , and product 
evaluation . As these three subsystems arc expanded in the studyi 
their interrelatedness is established in view of the total project^ 
that lS| in relation to the entire spectrum of project-related 
activities and objectives. 
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; o Evaluation or evaluative research is still in the questionable 

I 

I or developmental stages in many realms of education^ It is shown, 

ho^vever, that several unique characteristics of Title III activities 

i ■ 

tend to provide a reasonable basis for evaluative research and 

I 

! effective evaluation strategies. 
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I . ESKA TITLE III PROJECTS 



Projoct Proposal and A{>plication 

Title III of the Elenficntary and Secondary Education Act of 
19o5 requires that every initial project application contain an 
evaluation plan or strategy as an integral part of the proposal* 
U^en developed and implemented, these evaluation efforts should 
enable the applicant agency to ascertain the degree to which 
project objectives have been met* That is, these efforts should 
yield timely and relevant inforruation for use by decision makeis, 
information that shows. the extent to which project-related activi- 
ties (treatment) were effective in achieving objectives of the 
project. 

Furthermore, in subsequent applications for second and third 
grant awards the applicant agency must submit a report which 
displays the results of evaluation efforts. This report likewise 
is to include proposed revisions in the original evaluation plan 
or strategy affecting the remaining portion of the project. 

Project Evaluation Reports 

A written evaluation report must be submitted by each project 
at the end of each grant period. General guidelines for such 
reports arc as follows: 

First End-of-Grant Report 

This report should be submitted days prior to the end of 
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the first grant period along with the application for the first 
continuation grant. Primary consideration in the report should 
be given to changes which have occurred as a result of project- 
related activities during the first grant period. The goals of 
the report should be; 

(1) To present the results of Baseline evaluation which 
has taken place during the first year of project operation and 
which describes the context within which first year activities 
have taken place; 

(2) To specify the steps taken to meet project objectives; 

and 

(3) To describe as clearly as possible the degree to 
which project objectives were achieved during the first year of 
project operation. 



Sec o nd End-of-Grant Report 

This report should be submitted 60 days prior to the end of the 
second grant period along with the application for the second con- 
tinuation grant. Because project goals, objectives end activities 
often cannot be categorized on the basts of yearly grant periods, 
the second end-of-grant evaluation report should be cumulative, 

Tlius, the goals of thc^ report should be (I) to present the results 
oC any Baseline Evaluation which has taken place during the second 
year of project operation and which is rclcvart to second-year 
activities; and (2) to describe as clearly as possible the cumulative 
effects of the project In terms of all activities which have taken 
place > and all objectives which have been met during the first 
two years of project operation, 

4 
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Proiccl' Tcmlnation Report 

This report must be submitted along with the end-of-project 
budget report no later than 90 days following the project termi- 
nation date. Again, because project goals, objectives and activities 
cannot be categorized on the basis of yearly grant periods, the 
project termination report should bo cumulative and should take 
into consideration all project objectives and all activities under- 
taken to meet those objectives for the entire duration of the project* 
The primary goal of the report should be to present, in tne light of 
information gained from Baseline Evaluation , a full and clear 
picture of both successes in meeting project objectives and failures 
to meet project objectives. Needless to say, termination reports 
in which only project successes are prescpted, ii. which project 
successes arc couched it unrealistic terms, or in which project 
failures arc burled In euphemisms, do a great disservice to the 
progress of educational research and development- 

Projects lasting for three years arc required to submit First 
and Second End-of-Grant Reports as well as a Project Termination 
Report. Projects lasting for two years arc required to submit 
the First Eiid-of-Grant Report and Project Termination Report. 

Projects lasting for one year arc required to submit a Project 
Termination Report only* All project evaluators arc encouraged 
to include In their records documentations summarizing evaluation 
activities at three to six month intervals. Such sumir.arlcs greatly 
facilitate both the project decision-making procc^is and the writing 
of required evaluation reports. 
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3d Fomat: and Coasidoratlons for Content in Titl e III Report s 

Writeca evaluitioa reports should always reflect as completely, 
as possible both the project , valuation strategy and the outcome of 
application of that strategy. In effect, this tneans that the format 
of each evaluation report should l)e carefully developed in relation- 
ship to, specific project evaluation activities and outcomes such 
that all of those activities and outcomes are included in the report. 
Thus, the following list does not represent a definitive format for 
every Title III project evaluation report. It does, however, present 
some of the basic considerations germane to most evaluation reports. 
Writers of Title III project evaluation reports are cautioned not 
to use the list as a format for their reports, but to consider the 
list carefully from the point of view of a meaningful organization 
of the content In their reports. 

A. Research Results 

1. Define the scope of evaluation efforts in relatlCLi to 
project objectives. 

2. Describe the general research de8ign--tnis is, provide an 
overview* of the schedule of event; which constitute research efforts. 

3. Describe instruments used to collect data. Include in the 
Appendb to the report a copy of all instruments used, except 
standarcized tests. 

Discuss statistical ptoc.^durcs and methods. Include in 
the Appendix all tabulated raw data relevant to statistical com- 
putations presented in the body of the report. 

3. Discuss the results of the evaluation study in terms of 
educational and statistical significance as related to project 
objectives. 
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6. Present conclusions of the evaluation study and recommen- 
dations for future activities and studies. 



B. Personnel Responsible for Project i!vaUiation 

1. Dosciibe the qualification responsibilities of all con- 
sultants providing evaluation services to the project. 

2. Discuss the extent and kind of involvement for all those 
who are participating in project evaluation. 

C. Difficulties Experienced During the Course of Proiect Evaluation 

1. Describe any difficulties encountered in developing a project 
evaluation strategy. 

2. Describe any limitations and constraints relevant to data 
collection, analysis and interpretation. 

3. Discuss any difficulties relating to aval labillty/quallf t- 
cations of personnel needed for participating in project evaluation 
activities. 

4. Discuss or describe any other difficulties relevant to 
evaluation of your project. 

D • Kf fee Jiveness and Efficiency of Research Ef f orts in Relationship 
to Project Objectives 



1. Discuss the extent to which research efforts have provided 
useful information relevant to the impact of the project on students 
involved, (O on participating staff members, and (2) on the com- 
munity or area in which the project took place. 

2. Determine if the information provided through the evaluation 
efforts Justifies the expenditure for those effo'^ts. If more funds 
should have been provided for evaluation efforts, or If more funds 
will be needed during tho next grant period, describe the reasons. 

3. In light of project results to date, discuss evidence that 
indicates whether current project objectives should or shoild not 
be pursued during the rest of the project (for First and Second 
2vid-of-Grant Reports only). 

4. Report total estimated expenditures for evaluation efforts. 
Major resources and activities Include: 
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(i*) Project evaluation staffj 

(b) Consultants both from the school district (or districts) 
served by the project and from other sources; 

(c) Materials, including cost of standardized tests and 
other instruments; 

(d) Contracts for statistical analysis of data; and 
(c) Other resources and activities. 
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II. IiVAIJ;ATION of ESI-A title III PROJlvCTS 



Every Title III project is managed by a project director and 
a team of specialists who must frequently make decisions regarding 
resources used and activities undertaken, to maet project objectives* 
To make the most effective and fruitful decisions, and thus to 
provide the best possible assurances for project success, the project 
director and his team of specialists must obtain, maintain and 
interpret accurate, up-l. J^te bodies of information relevant to 
project design, project development, project implementation and 
project successes/failures • 

Evaluation Defined in Context of Title III Projects 

The process of gaining and interpreting such information is 
termed "evaluation"* More accurately, evaluation may be defined 
as the process of delineating, collecting and interpreting infor- 
mation for distribution to those involved in the dec is iC’>-making 
process* 

Delineating refers to outlining parameters for information 
to be collected. Such outlining should clearly specify the kinds 
of data being sought in terms of content, scope, reliability, 
validity, and appropriateness and relevancy of source* 

C ollecting refers to' gathering and compiling all available 
data froTi established parameters with precision and in useable form* 

Interpreting refers to explaining the statistical and 
educat ioial significance of collected data in a form that is und<3r- 
staivJable by those to whom the report is directed* 

Distrihvitlon refers to providing appropriate and necessary 
written reports to those involved in the decision-making process* 
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Genera] Evaluation Functions and Cafcgorics 



Although detailed procedures for evaluation will vary widely 
froia project to project, it is possible to outline a set of general 
evaluation functions which are apropos to every Title III project. 
The functions fall into three categories: Baseline Evaluation , 

Process Evaluation , and Product Evaluation . It should be observed 
here that the delineating, collecting, interpreting and distributing 
activities explained in the definition given above apply to each 
functional category within the overall evaluation process, so 
that an outline of general evaluation strategy night i;2 developed 
around the following franieworkr 

Baseline 
Evaluation 
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Delineate 

Collect 

Interpret 

Distribute 



Delineate 

Collect Process 

Interpret Evaluation 

Distribute 



Overall 

Evaluation Process 



Delineate 

Collect 

Interpret 

Distribute 



Product 

Evaluation 
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Baseline Evaluation 



The first procedure in an overall evaluation strategy should 
be Baseline Evaluation i Baseline Evaluation may be defined as 
the delineation, collection, interpretation and distribution of 
all information needed (1) to establish data bases which provide 
input into needs assessment activities (those activities which 
result in a clear knowledge of the educational needs of the agency 
in which a Title III project will be developed); (2) to provide a 
benchmark against which to measure future successes/failures of 
project activities; and (3) to furnish data for Identifying, 
describing and defining the context within which change is expected 
to occur as a result of project activities* In effect, Base line 
Evaluation provides that kind of information without which it 
would be impossible to measure any progress made toward meeting 
project objectives i Such Information describes conditions as 
they exist before project treatment; and unless these conditions 
are accurately described before project Implementation, it will 
be Impossible to describe changes effected by project implementation* 
Baseline Evaluation is, therefore, essential to the overall evalu- 
ation process; and every care should be taken to insure that 
information gained during the Baseline Evaluation process is valid, 
appropriate and veil documented* 

Process Evaluation 

After the project has been Initiated, simulated or implemented, 
and throughout the course of the project, managciftcnt will have to 




make many important decisions regarding project resources and 
activities • 'Ihe f-nction of Process Evaluation is to aid manage- 
ment in making day-to-day opfirational decisions. Essentially, 
Process Evaluation may be defined as the delineation, collection, 
interpretation and distribution of infoj.ination needed to make 
decisions which will keep project resources and activities directed 
toward fulfilling project objectives. Thus, Process Evaluation 
assesses the fulfillment or non-fulfillment of those key steps 
which, having been carefully plannea during project development, 
are being taken to meet project objectives. The result of such 
assessment should be to make decisions regarding subsequent courses 
of action to be taken, and resources to be used, toward fulfillment 
of project objectives* 

Product E v aluation 

As planned steps to meet an objective or set of objectives near 
completion, project management should be prepared to undertake 
Product Evaluation . Product Evaluation may be defined as the 
delineation, collec^ion, interpretation and distribution of infor- 
mation n'^eded to determine (i) whether or not a project objective 
or set of objcctivss has been met, and (2) the extent to which a 
project objective or set of objectives has been met* Information 
resulting from Product Evaluation should permit project management 
to gain a clear overview of project progress in terms of project 
objectives . For this reason, due consideration from the very 
outset of project design and development should be given to stating 



project objectives in measurable terms • The less measurable pro- 
ject objectives are, the less probable it will be that effective 
Product Evaluation can take place; and, unless effective Product 
Evaluatio n can take place, project management will have no rational 
foundation on which to base those key decisions which will affect 
the success or failure of the project as a whole, as well as the 
continuation of the project, either in part, or as a whole. 

Role of Evaluation In ESEA Title 111 Proiects 

The following points should be taken into consideration with 
regard to evaluaticn of Title III projects: 

1. Evaluation is an ancillary process; the main purpose 
is to provide project management with accurate Information so that 
management can make critical decisions regarding project resources 
and activities. Evaluation Is not the decision-making process 
itself; rather, it is a service which facilitates the decision- 
making process. 

2 . Paseline . Procot s and Product evaluation procedures 
should be scheduled in proportional importance with major project 
activities so that the evaluation strategy can be developed in a 
reasonable perspective with the project, and not vice-versa . At 

no time should evaluation become the dominant activity of the project. 

3 . Finally, careful consideration should be given to the 
format of every evaluation report to insure that all activities which 
have taken place and all objectives which have been met during the 
period covered by the report are appropriately presented in the report. 
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III. KVAI.UATION IN' PERSPECT I VE WITH OTHF.R PROJECT RHLATHD ACTIVITIK S 

Figure I attempts to place Base 1 Ine > Process and Product 
K valuvit ion in perspective with other major project activities. 
Project-related activities are numbered from 1 to 17^2 to indicate 
activity flow over time. Figure 1 by no means depicts the relation- 
ship between all evaluation activities and all major project 
activities. For the purposes of proposal development and the 
development of an evaluation strategy, however, the model does 
in general indicate the role and place of Base lino , Process and 
Product Evaluat ion activities. This model of an evaluation system 
for ESFA Title III projects portrays evaluation activities as three . 
major subsystems » each consisting of a block of project-related 
activities^ forming part of the overall Title III Project System 
and fitted within the sequentially related activities of the 
entire project. 

In developing this model several different models and 
descriptions of the ^'systems approach" were utilized, including 
thc‘^'‘'odel of Systematic Approach to System Development^' of R. C. 
Hopkins (Figure 2), the ''Model of General Steps in Problem Solving 
Using System Analysis Tools" of Corrigan and Kaufmann (Figure 3), 
and the "Model of the Systems Approach" used by Launor Carter 
(Figure A). Each of these models will be used later in the section 
of this study devoted to an evaluation of the model given in Figure 
1. For our present purposes, however, let us "take a walk through" 
Figure 1 by relating the steps outlined in this model to the steps 




involved in a systematic approach to system development, as 
indicated by R, C. Hopkins in Figure 2, This approach will enable 
the reader to see Che "systems” terminology couched b'l^nind each of 
the labeled and sequential steps in the development of project- 
related activities in Figure 1. 

Baseline Evaluation Subsystem 

Activity 1 in Figure 1 consists of recognizing that objectives 
set by the local educational agency arc not being met* This recog- 
nition is based on the answers given to the fundamental question, 
"What is the local educational system supposed to do and vjhy?" The 
answers to this question comprise a list of the needs to be fulfilled 
by the LEA system, the uses to which the LEA system is put, and the 
purposes for the existence of the LEA system* This list is then 
analyzed for duplications, similarities, analogies with other 
systems, nonpertinent items, state-of-the-art implications, and 
fundamental meauings and relationships. 

Activity 2 constitutes the first formal step of Baseline 
Evaluation -- that is, the determination of specific problems and 
needs resulting from lack of fulfillment of objectives* Such 
determination should be made on the basis of available data which 
have been delineated, collected, interpreted and distributed in 
Accordance with the definition of Base line Evaluation given above* 

In other words, the functional objectives of the LEA system are 
synthesized, by way of induction, from the analyzed list of ures, 
needs and purposes of the LEA system. 
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CXice a determination of problems and needs (needs, uses and 
purposes synthesized into functional objectives) has taken place, 
Activity 3 can be undertaken. Essentially this activity consists 
in comparing the needs of the local educational agency with an 
existing list of state critical needs for the purpose of determi^ 
nlng eligibility for Title III funds. 

Should the LEA be eligible for Title III funds, Activities 
4 and 5 can take place. Determining the strategy for treatment 
comprises the development of conceptual and specific requirements 
for the LEA system, as well as the expansion of the specific 
(performance) requirements by the development of quantitative 
specifications for these specific requirements. The conceptual 
requirements for the LEA system are developed by answering the 
question, "What docs the system V.ave to do to fulfill Its objectives?". 
The specific (performance) requirements for the LEA system arc de- 
veloped through an expansion of each conceptual requirement by 
asking, "What does the system specifically have to do in a detailed 
physical (or other appropriate) sense in order to fulfill that 
requirement?". These specific performance requirements developed 
for the LEA system correspond to the specific design requirements 
in weapon systems, computer systems, etc. 

The detailed requirements are then grouped together into 
subsystems or components. The subsystems are developed by successive 
partitioning until manageable chunks can be assigned to particular 
design and development groups, This activity greatly facilitates 
an analysis and derivation of structure and techniques and of 
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detailed design of the LEA system, a) I of which should eventuate 
in the development of an initial proposal that contains (1) a 
well-balanced overall and preliminary conceptual design of the 
LEA system, and (2) sufficiently quantified and measurable objectives 
upon which to base an effective evaluation of the protect. 

Activity 5 in Figure 1 is especially important, for it is 
at the stage of project development that measurable objectives 
for the project should be formulated. It is, after all, the 
intrinsic value of efforts included in Activity 5 that will 
determine the approval or disapproval of the entire project. 

Activities 3 and 6 of Figure 1 correspond to environmental 
considerations in the sequence of steps Involved in Hopkins 
systematic approach to system development. The physical, legal, 
organizational, sociological, psychological and economic environ- 
ments surrounding the LEA must be taken into consideration in the 
overall proicct proposal development. More specifically, it is 
the economic environment of Title III funding, along with a com- 
parison of the needs of the LEA with an existing list of state 
critical needs, that determines eligibility for project funding 
under Title III (Activity 3) and approval of the project proposal 
(Activity 6). 

After the proposal has been approved, the establishment of 
[baseline information can continue. Basel Inc information . In the 
context of Activity 7, Is that information vhlch provides bench- 
marks against which to measure successes/failures of project 
activities. Activity 7 be compared to the modeling, simulating 




and testing of the preliminary and conceptual design of the system* 

In conjunction with all activities to this point, Activity 7 com- 
prises the first major evaluation, fccdbick, and modification loop 
in the entire sequence of activities undertaken. More specifically, 
as Figure 1 indicates, Activities 2 and 7 together comprise Base 1 inc 
Fvaluation , which, in tenns of the Hopkins model (Figure 2), includes 
all activities croji the original listing of uses, needs and purposes 
to the modeling, simulating and testing of the LEA system design 
(See Figure 5). 

Such information as that contained in Baseline information 
should be collected and documented be for e any attempt is made to 
fulfill project objectives. Although sc^veral activiti occur 
between Activities 2 and 7, and althougli Activity 7 may occur during 
a project at ary time just prior to the initiation of planned steps 
to fulfill an cbjcctive, Activities 2 aid 7 constitute Baseline 
Evaluation. 



Process Evalua ti on Subsystem 



Base l ine information has been established such that 
there exist bcr.climarks against which to measure the first major 
sot of project activities, refinement cf the strategy for treatment 
(Activity 8), project implcmcntatioii (Activity 9) and execution of 
steps to fulfill project objectives (Activity 10) can take place. 
Activities 9 and 10 shoulo have been well-planned during proposal 
development; and, as each step is taken, it should be assessed to 
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oetermine i£ the outcome is desirable or undesirable in terms of 
progress made tovard meeting objectives • Methods of determining 
the outcome of each step taken will vary widely but should i in all 
cases, provide that kird of information which, when compared with 
Baseli n e information , will make it possible to determine whether 
or not the outcome of the step is desirable. As indicated in 
Figure 1, Activity 11 and Decision 12 constitute Process Evaluation * 
Should the outcome be undesirable (Activity 13*1) the step can be 
modified, executed and rc-evaluated until the desired outcome Is 
attained or until other previously planned steps arc undertaken. 
Should the outcirae be desirable (Activity 13.2) the next planned 
step to meet objectives cf*n be undertaken and this cycle can con* 
tinue until completion of the planned steps to meet objectives. 

Activity 8 of Figure 1 corresponds to the activities of 
subsystem development, subsystem conceptual design, component 
design, and system dcjlgn finalization and Integration, as defined 
In the liopkins model (Figure 2). Activity 9 (implement project) 
and Activity 10 (execute steps to fulfill ob jectlve (s) ) do not 
appear In the Hopkins model as such. In actuality these activities 
represent Implementation of the project by the successive execution 
of the steps necessary ts fulfill project objectives prior to any 
Frccess {^valuation . 

Activity U and Decision 12 of Figure 1 comprise system 
testing tn th; Hopkins model. As ouch they constitute the second 
major eva lua»:ion, fcccback and modification loop In the sequence 
of activities underrakoii relative to Title III project development. 
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As a matter of fact. Activity 13*1 of Figure 1 is implied in Process 



Kvaluat ion , for it involves any modification required as a result 
of Process Evaluation * 

Product Kvaluntion Subsystem 

Upon completion of all planned steps to meet a given objective 
or set of objectives, it will be necessary to determine the extent 
to which the objective (s) have been met (Activity 14), Taken to- 
gether, Activity 14 and Decision 15 of Figure 1 constitute Product 
Evaluation » Should the objective (s) be met to the satisfaction of 
project management, planned project activities can continue (Activity 
l6.2) and activities undertaken to meet the objectlve(s) should con- 
tinue for the purpose of validating initial outcome of product 
evaluation (Activity l6>3). 

Should the objectives not be met, management must decide 
either to repeat or not to repeat activities relevant to the unmet 
objective(s) (Decision l6«l)i A decision not to repeat such activi- 
ties will bring about the necessity of preparing for future project 
activities in light of the fact that one or more of the project 
objectives have been terminated (Activity 17.1)i A decision to 
repeat such activities calls for consideration of both modification 
of the present project strategy and formulation of alternatives for 
future project activities (Activity 17.2) ■ 

A''.tlvlty 13.2 in Figure 1 corresponds to system Implementation 
in the Popklns model. Activities 14 through 17.2 in Figure 1 are 
not depicted in the Itopklns model at all. Activity 14 and Decision 
'5 of Figure I comprise testing and evaluation of the cntij'2 system 
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after it lias been implemented, a testing of the implemented system 
and overall evaluation of the system as impleTiier.ted . This final 
and major evaluation^ feedback and modification loop is not shown 
In the Hopkins node I, but it is this loop that constitutes final 
product evaluation (See Figure 5). In tlie Hopkins sense, this 
ev luation, feedbac,< and modification loop would consist of an 
analysis of the existing system as a part of the preliminary steps 
In the systems analysis spectrum of activities. 



Interrelationships Among Evaluation Subsystems 



Tlie preceding discussion p' iateU out (I) that evaluation may 
be protrayed as a set of subsystems forming part of a larger system, 
namely, the spectrum of activities Included in Title III project 
proposal, approval and development, and (2) that the three identified 
evaluation subsystems in fact constitute three major feedback end 
modification loops. Figure 3 consists of a model of the general 
steps in problem solving using system analysis Cools as proposed 
by Corrigan and Kaufmann Ln their paper entitled. Th e Tools and 
Stops of the System Approach to Education prepared for Operatien 
PEP participants. As this model clearly Indicates, evaluation 
actually occurs, or should occur, at each and every step in the 
design, development and engineering of any systcr.. To indicate 
this fact of continuous and continued evaluation activities In 
Figure 1, however, would have rendered the model in Figure 1 
entirely too complex and unwieldy for its projected users ■ Al- 
though Figure 1 does not pictorial ly display »^he continuing nature 
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of cvaljatioA activities, it is hoped that the disciission has 
brought this aspect of evaluation to light# 

Figure I sherds Baseline F.valu^tion as consisting solely of 
Activities 2 and 7; the discussion, ho\7evcr, indicates that Baseline 
Evaluation , in reality, includes all project-related activities from 
Activity 1 through Activity 7. Similarly, Figure 1 shen^s Process 
Evaluat i on as consisting solely of Activity 11 and Decision 12; 
again, however, the discussion explains that Process Evaluation in 
fact includes Activ\.ties 8 through 13.1. The same is true for 
Product Evaluation , which Figure 1 indicates as Including Activity 
14 and Decision 15 only. Realistically, however. Product Evaluation 
includes all project-related activities fror^ Activity 13.2 to Activity 
17.2. Thus, from this discussion, it is clear that evaluation 
activities in connection with Title III projects do occur as a 
continuous function over the sequential time-related activities in- 
volved in Title III project development, and that the evaluation 
subsystems in Figure I are highly integrated within the sequence 
of activities ard with each other* 

Furthermore, although evaluation activities must occur at 
each and every stage of system design, system development and 
system implementation, it is trua that there are certain points 
\n this sequence of activities that can be termed "major evaluation, 
feedback and modification loops". Although t' i model in Figure 1 
does not portray this fact vividly, again the discussion above 
confirmed the f.ict that the three idontified evaluation subsystems 
do represent three major evaluation, feedback and modification loops* 
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Whereas the Corrigan & Kaufmann model (Figure 3) emphasizes the 
continuing nature of evaluation activities, it does net indicate 
where major evaluation activities should occur* Figure 4, which 
represents another model of the systems approach, does indicate to 
a greater extent that feedback to previous steps to investigate the 
possibility of revising needs, objectives, constraints, alternatives 
and implementation methodologies can and should occur at certain 
strategic points in the development of a system. 

In the previous discussion it was indicated that the three 
evaluation subsystems, representing major evaluation, "ccdback and 
modification loops, could be identified and related to certain steps 
in the Hopkins model quite readily. Specifically, Kascltnc Evaluation 
was shown to correspond to the ''Modeling, Simulating and Testing of 
System Design"; Process Evaluation was shown to correspond to "System 
Testing"; and Product Evaluation , as was indicated, does not show up 
in the Hopkins model, but would occur after "Implementation", lo 
short, Baseline Evaluatio n, Process Evaluation , and Product Evalu * 
atlon correspond quite clearly to what might be called "Testing 
Preliminary System Design", "Testing Integrated System", and "Testing 
and Evaluating the Implemented System" In Hopkins' terminology. 

SuTnary and Conclusions 

Kvalua i ion of the Model of the Evaluation System for KfiHA Title TIT 
Pro ject .n 



How well does Figure I provide a sound theoretical basis for 
the evaluation of Title III project development? How well docs it 
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describe and explain evaluation in the perspective of Title III 
project activities? After all, these are the primary purposes of 
any model, namely, to describe, explain, and provide a sound theo- 
retical basis for the system being modeled. How well does Figure 1 
meet the objectives and purposes for which it was developed and 
which were set forth in the introduction to this study? 

Clearly, Figure 1 does not coincide step-by-step with any of 
the models developed for a systems approach to system development. 

In other words, there is not a one-to-one relationship between the 
17.2 steps in Figure 1 and the steps delineated in the Hopkins model 
01 any of the other models. This does not mean, however, that the 
correspondence between Figure 1 and other system models is thereby 
minimized. Actually, Figure 1 relates very well with most other 
systems approach models. 

Figure 1 clearly indicates the general role and position of 
the three evaluation subsystems within the overall context of Title 
III project development activities. It further relates the three 
evaluation subsystems to each other, as indicated in the previous 
section of this study. Although Figure 1 appears to be highly linear 
and eequential in nature, reflc:ting the sequential aspect of project- 
related activities, this appearance is due primarily to the fact that 
the intended users of this model (Title IIT project evaluators) clearly 
need a model that structures and organizes project-related activities 
sequent Lai Ly. Obviously, as is clear from an examination of Figure 
3 nnd Figure 4, project evaluation, like any other evaluation system, 
involves a highly interactive organization of activities, many of 
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vhicJi in turn are highly iterative in nature. As indicated earlier, 
to have specified the truly interactive and iterative nature of 
project evaluation would have co:;iplicatcd Figure 1 to such an extent 
as to rondo: it too cumbersome for its intended users ■ 



Rational ilasis for Effective Evaluation of Title III Projects 

As mentioned in the Introduction to this study, evaluation or 
evaluative research is still in the questionable and developmental 
stages in many realms of education. There are several unique 
characteristics of Title III projects, however, which tend to pro- 
vide a reasonable basis for effective evaluation strategics and thus 
for evaluative research. One of these characteristics is the fact 
that one prerequisite for any Title III project is that there be 
clearly defined objectives. These objectives, furthermore, must be 
behavioral in nature and truly measurable. Thus, through a clear 
delineation of the rudimentary elements of the project the foundation 
and necessary framework for subsequent evaluation is established. 

Secondly, all Title III projects are supposed to be innovative/ 

exemplary. This aspect tends to differentiate Title III programs 

\ 

from on-going or traditional programs. It further places more 

\ 

emphasis on the evaluative aspqc^ or activities associated with 
the project, for any innovative or^xe^lary project /system must 
necessarily be subjected to more rigid cvaLuatio^procedurcs , 

A third characteristic of Title III projects thTat^tends to 
provide a rational basis for evaluation of these projects is^th'et 
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Melinite parameters are established in terms of the length of pro- 
jects, with a maximum length of thirty-six months, in terms of 
funding approved for the project, and other limiting constraints 
that arc clearly known by the project evaluators. All of these 
pre -defined parameters provide precise and clear answers to same 
of the questions project development and project evaluation personnel 
must ask. Such parameters solve some of the system analyst’s or 
system developer’s questions concerning environmental considera- 
tions and constraints, as well as define the inputs, the outputs, 
and the boundaries of the project/system to be developed. All these 
characteristics of Title III projects collectively tend to provide 
a rational basis upon which an effective evaluation of Title III 
projects can take place. It is hoped that an understanding of 
the evaluation system depicted in Figure 1, which indicates the 
general role and position of evaluation in perspective with other 
Title III project related activities, will provide a valid rationale 
for systematic evaluation of Title III projects and Title III project 
related activities. 
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What, then, Is systems analysis as it Is used In military and space to 
About two years ago the Xatlonal Security Industrial Association formed a 
Working groups to examine the way In which military and space technology : 
applied to the many problems In civilian socletyi One of the groups deal 
systems approach to cducatlca. ’'f Ihls group formulated wight basic points 
felt charactcrl7J*d the typical application of the systems analysis to trn 
Lems, and suggested, that the extension of this approach to civilian c<3ucV‘ 
help significantly In Inprovlng our public education. Each of the eight 
descclbed In conjilderabl^e detail. The summary sentences for the points ve 
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1. State the real NEED you are trying to satisfy, 

2, Define the educational OSJECTIV'ES which wilt contribute to satis fylr 
need. 

Define Cho?e real world limiting CONSTRAINTS which any proposed sys’ 
Generate miny different ALTERNATIVE systems, 

SKIXCT the best alternative (s) by careful aralysls, 

IMPLEMENT the selected a3 tcrnatlvc (s) for testings 
perform a thorough EVALUATION of the cxporlnontal system, 

8, Rased on experimental and real world results, FEEDBACK the rcnulred 
and contlnic this cycle until the objectives have been attained. 
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While these -light points nay be typical of the activities In systenr n*' 
list does not adequately cr.,\voy the dynamic and continual, y Iterativ,* or L 
acter of systems analysis. The diagram above cnphaslr.os the ccntlnj.u o:.- 



acter of systems analys 

each step, the f< cdbock to previous stops, .iml the consideration of varN 
and limitations ^hlch influence the decisions made at each point, 
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